Negotiation-tools in CSCL-scenarios: do they have a valid use?

Within discursive knowledge construction, students are expected to negotiate within their collaboration as soon as they face opinions, concepts, or meanings differing from their own. Therefore, negotiation has become a central issue of CSCL research. In an experimental field study with 16 groups of 3 individuals each, we examined whether the demand to use technically supported communication (e-negotiation) within an asynchronous and spatially distributed setting has a positive influence on group discussions and knowledge integration. Our results indicate that the implementation of e-negotiation is in fact advantageous, but does not automatically lead to a successful result. Employed intensively, enegotiation allows group members to emphasize incompatible ideas and therefore offers advantages especially in creative problem-solving processes. Introduction One of the most crucial challenges within asynchronous CSCL is the encouragement of students to refer to each other and to their contributions. In discursive communication (Herrmann & Kienle, 2008) referencing to the concepts of others indicates the degree of collaboration (Stahl 2003) in contrast to situations where a number of students work on the same problem but in solitude. Within discursive knowledge construction, students are expected to negotiate within their collaboration (Dillenbourg, 1999) if they are faced with opinions, concepts or meanings which differ from their own. Therefore, negotiation has become a central issue of CSCL-research. Negotiation is the process where differing perspectives are related to each other and where a convergence between unconnected or incommensurable ideas is pursued in the course of decision making. Negotiation is especially relevant in the case of problems or tasks where the solution is not determined in advance and where a wealth of ideas and contributions is needed to find a good solution by building synergy and pursuing convergence. Negotiation can be a subject of analytical work as well as a focus of intervention strategies. From an analytical viewpoint, the negotiation of the meaning terms or concepts is observable. Moreover, it can be observed that a breakdown experience (Stahl, 2000) with respect to mutual understanding leads to negotiation. The negotiation of meaning is considered as a fundamental basis of the development of mutual understanding and sense making. Therefore negotiation can be an analytical category which helps to detect the occurrence of mutual knowledge construction. On the other hand, negotiation can be an orientation for intervention strategies which foster collaborative knowledge construction. Conversation schemes or scripts (e.g. for argumentation) can be offered or prescribed to guide the way in which students relate their contributions to each other. Furthermore, the technical features of CSCL can be designed to provide a scaffold which represents the guidance through the conversation schemes. The guidance can be focused on the type and sequence of conversational chunks (such as proand contra-arguments) or on the support of decision making, e.g. by offering a voting mechanism. We have developed CSCL-prototype called Kolumbus2 (Herrmann & Kienle, 2003) which is strictly oriented towards the support of interactive, mainly asynchronous communication and the conveying of the context which completes the communication processes. This prototype includes a negotiation-function which completes the work with shared material and the annotated conversational threads with possibilities for voting; votes can optionally be commented upon by the voters. The basic assumption behind this feature is that the availability of negotiation support together with a clarification about how to use it has a positive impulse: The students of a learning group are asked to make a decision in the sense that they all agree to be co-authors of certain statements which represent the shared results of the group. It is assumed that this decision making promotes a phase of convergence in which the multiplicity of divergent ideas, which are produced in the context of thought provoking tasks, are integrated. The Kolumbus2 prototype was tested in several practical cases. These case studies revealed that the negotiation function was used and accepted. Several improvements were proposed by the participants. It became clear that the usage also depends on the planning of the collaboration process and on the influence of a facilitator (Carell et al. 2005). However, up to this point we had had no insights into whether the negotiation and voting process just led to some non-reflected pragmatic decisions or whether they were accompanied by deliberate knowledge integration. We assume a positive effect from the voting-based negotiation procedure: those students who have to make a decision between “accept” or “reject” may be motivated to develop a clear understanding of how their own ideas are related to those of others. They will be able to identify those contributions which provoke the most divergent reactions. The students attempt to overcome this divergence may result in a higher degree of convergence. To validate these assumptions, we designed an initial experiment with 16 groups, each made up of 3 participants, to achieve first results whether the degree of convergence is influenced by the request to use a negotiation tool and whether it can be measured. On the basis of this initial experiment it is possible to describe the requirements for the underlying task, the procedure, the instructions and the underlying hypotheses have to be specified within an ongoing investigation of the relationship between knowledge construction, and the availability of negotiation functionality. One of the crucial questions is whether the availability of a negotiation tool really does promote convergence and knowledge construction or whether it is just a means which makes it possible to systematically observe that those students who are willing to negotiate and to integrate their knowledge are using this functionality. The following section gives an overview of related results of research on negotiation in the context of CSCL. The third section describes the experimental design. It is followed by a description of the results and the concluding discussion. Research on negotiation and the consequences for negotiation-support One of the most detailed overviews on the roots and relevance of negotiation is given by Stahl (2003) who refers to the background of negotiation within CSCW and to the underlying concepts of collaborative learning. Within CSCW-research, negotiation is mainly important for Group Decision Support Systems (GDSS, deSanctis & Galuppe, 1987, Vogel et al., 1987) and is usually accompanied by possibilities for voting which allows the users to prioritize their favorite options within a set of available choices. Stahl contrasts this kind of decision making with the role of negotiation within theoretical frameworks of collaborative learning such as the small group process, social constructivism, distance education, distributed problem-based learning, distributed cognition, cultural-historical activity theory. He outlines that negotiation has its relevance within these theoretical approaches with respect to knowledge building. He concludes that “the concept of negotiation as voting seems inadequate for CSCL. In particular, the negotiation of what is to count as new shared knowledge for a group engaged in collaborative knowledge building has different characteristics from other forms of group decision making” (Stahl 2003). We comply with this suggestion since we emphasize that the process of voting itself is not the arena of negotiation but the accompanying discussion threads. Therefore a voting mechanism has been combined with the possibilities for commenting and for annotated discussion threads of Kolumbus2. We call the type of negotiation, which is triggered by the request to accept or reject a proposal, e-negotiation. The design rational is that repeated voting, which is suggested to the students, is a trigger which increases the degree of knowledge sharing and integration. The experiment which is described in this paper is a first attempt to validate this assumption. Within his review on negotiation research, Stahl (2003) also analyzes the strengths and weaknesses of an early version of Kolumbus2 (Stahl & Herrmann, 1999). Based on his analysis, he developed another negotiation-supporting CSCL-system: BSCL. Kienle (2007) has conducted several case studies to understand how negotiation support has to be designed, and she (Kienle, 2007) compares BSCL with Kolumbus2 with respect to practical experience and user feedback. The improved negotiation mechanism of the current version of Kolumbus2 refers to Kienle’s work. A specific feature of Kolumbus2 is that the result of voting leads to an automatic activation of certain functions by the system. This approach was developed together with Wulf, Pipek and Pfeifer (2001), who have focused their work on negotiation between two roles, where – for example – one person requires a certain access right and the other person can agree to it or reject it. If the negotiated item such as a piece of text or a document is accepted, Kolumbus2 changes its status in accordance with the proposal which has been made by the initiator of the negotiation. Possibly, a new version of the item is stored, the group of recipients (who can read it) is enlarged or new co-owners are added. Co-ownership (Prilla & Ritterskamp, 2006) is the typical way of documenting so that several students consider an item within Kolumbus2 as their shared contribution. The early work of Dillenbourg and Baker (1996) on negotiation spaces was not taken into account for the development of Kolumbus2. However, our approach complies with their requirements to support symmetrical interaction and to be aware of various dimensions such as the degree of flexibility, systematicity etc. Another CSCL-discourse deals with the phases and sequences into which negotiation is embedded or consists of. These phases can either be used as categories which help to analyze CSCL-based discourse, or they can be used to define scripts which are implemented into the system to scaffold the students’ interaction. An influential contribution was made by Gunawardena et al. (1997), who provide an interaction analysis model with five phases: statement and application of newly constructed knowledge, testing, negotiation, dissonance, sharing and comparing. Beers et al. (2005) refer to five other but similar negotiation primitives: contributing, verifying the understanding of another’s contribution, clarification, acceptance/rejection of a contribution and explicitly stating the own position on a contribution. These primitives serve as a basis for a negotiation tool that “... coerces the users into exploring each other’s perspectives to augment the negotiation of common ground (Beers et al. 2005, 625).” The tool was tested in an experiment. It was assumed that coercion leads to a higher Figure 1. Screenshot of the negotiation-tool number of negotiations per contribution. However, the control groups produced an even higher number of contributions. This observation correlates with Dillenbourg’s (2002) reasoning on the problems with overscripting. The Kolumbus2-protoype does not apply any scripts but offers a high degree of flexibility since we do not want to test the effects of scripting but of weaker types of intervention which combine the offer of certain functions with the suggestion to use them. We differentiate between three phases (Herrmann & Kienle, 2003): 1) work with one ́s own material and research results, 2) work with the material and results of others and 3) collaboration which includes threaded discussion and negotiation. The phases are not sequenced but the students can flexibly switch between them. Also, the negotiation-function is designed with a high degree of flexibility with respect to the dimensions which are displayed in table 1. Table 1: characteristics of CSCW-related negotiation general and KOLUMBUS (Kienle & Herrmann 2004) characteristics general (KOLUMBUS) aim of negotiation relevance or appropriateness of data, decision about further activities, decisions about the configuration of the system, joint editing decisions types of proposals access rights, authorship, editing functions (insert, copy, paste), rating, ... no. of participants > 1 (ca. 5 – 20) discussion and commenting discussion threads related to votes, proposals or to the negotiation process as a whole options of voting accept, reject, abstain, counter proposal, “Lets talk” visibility of voting Secret vs. comprehensible, anonymous vs. assigned to persons, statistical information about the negotiation process vs. result oriented information mode of voting One vote per person per proposals vs. votes can be repeated and changed . Voters can be allowed to flexibly alter a) their votes during the negotiation’s time span, b) the negotiation deadline, and c) the recipient of the negotiation (only at the beginning but not during the negotiation process). Votes can be commented on and the comments can be continuously extended. Therefore, voting and discussion threads are highly interwoven. The negotiation is started if a participant tries to activate a function which affects the right of others. For our experiment, we referred to the function which converts others into co-owners of a textual statement. The explanation of the co-ownership clarifies that co-owners have to identify themselves with the meaning and the content of the co-owned statement – they agree with a kind of co-responsibility for the content. If some-one invites others to become coowners, a negotiation is started (see figure 1). Those who are invited have the right to accept or reject the invitation or to abstain. The initiator’s proposal is counted as an acceptvote. The vote can be altered but the negotiation proposal cannot be withdrawn (in distinction from BSCL, Stahl 2003) since we want to avoid the votes and comments made by others being deleted and therefore not valued. The diagram of figure 2 gives an overview of the possibilities of the negotiation procedure. Figure 2. Possibilities of the negotiation procedure Experimental field study In the following we present the quasi-experimental field study carried out by us. Here we understand this study as an “explorative study of an experimental nature” which should contribute to an empirically proven formation and precise explanation of hypotheses within the area of investigation. Aim and assumption of the study The aim of the study was to find out whether the offer and demand to use technically supported negotiation (enegotiation) within an asynchronous and spatially distributed problem-solving process leads to • groups bringing together and integrating their discussion processes better (assumption 1). Investigations show, for example, that virtual groups have hardly any problems collecting their ideas within the framework of brainstorming. The difficulties lie, above all, in the next phase of convergence formation where it comes to the systematic selection of the compression or intertwining of ideas. Not least due to time pressure, a good idea will often suddenly be accepted as “the solution” without a unanimous group consensus being reached (see Carell 2006). • a more transparent and more easily recallable group result (assumption 2). • a more homogenous individual perception of the group result (assumption 3). Especially with asynchronously lead discussions, those studying will often receive the contributions of others insufficiently, do not have them present and can accordingly only insufficiently incorporate them into their own formulation or into the description of the group results. It is much more usual that the participants ́ contributions stand together unconnected. We assume that an explicit computer-supported negotiation process enables students to become more intensively familiar with the negotiation content which in turn becomes more current to them before they react to it. Furthermore, we assume that the demand to use e-negotiation provokes a more controversial yet focused discussion (assumption 4). At the same time, we wanted to determine within the framework of a qualitative evaluation used during the course of the group discussion at which points e-negotiation can be started and what effect this has upon further discourse (question 1). Setting and Data Collection Setting To examine the afore-mentioned assumptions, we carried out an experimental field study from June 2007 to October 2007. The study took place in two waves. In total 48 individuals voluntarily took part in the experiment (28 male and 20 female). For this experimental field study the following setting was organized: • Groups: 18 groups, each comprising 3 individuals, were formed. Each of the 18 groups was formed by first assigning a person with deep knowledge of the learning system being used (www.kolumbus2.de) and who, in the role of “power-user”, was available to answer questions regarding the use of the system. Second, the other group members were randomly assigned. As more males than females took part in the experiment, a weighting of men and women during random selection was not deemed necessary. All participants had been made familiar with the learning environment Kolumbus2 through a one hour introduction. Table 2 shows an overview of the male / female grouping. Two groups were not included in the evaluation because one member in each group neither took part in the communal group project, nor filled in the final questionnaire. Therefore, the evaluation of the experiment relates to a total of 16 groups. Table 2: Group formation according to gender 1 female 2 male 2 female 1 male 3 female 0 male 0 female 3 male Number of groups 5 6 1 4 • Task: The group, made up of 3 individuals, received the task to look at the website www.pepysdiary.com and to develop together a concept on how its essential idea could be transferred to university teaching events. Pepys, on whom the website is based, lived in 17th Century London and kept a diary over many years. Every day, one page of the diary is published on the internet site. The first entry in the diary from 1st January 1660 was published on the 1st January 2003 and every day since, a new entry has been published. To complement the diary site there is also an encyclopedia which contains historical facts about people, places, things and events, more in-depth articles about specific aspects of Pepys ́ diary, as well as information on the latest activities of the internet site etc. Therefore Pepy’s diary has developed into a successful learning environment. The task for the participants in the experiment was to make a proposal of how elements of this example can be transferred to improve the organization or electronic support of university courses. This task was chosen as such because there is no pre-defined solution which one can find or name. Through this, a higher variety of expected results or contributions was produced, which, in turn, leads to the chance being higher that the contributions are firstly heterogeneous and only converge later on in the process. For this experiment, it would not have been appropriate to set a task which produced homogeneous contributions from the start. One intervening factor which should be noted here is the influence of the used system – in this case Kolumbus 2. Such a system can, in the context of our chosen task scenario, lead to a common context or framing within which the contributions relate to each other and therefore become more uniform. • Time frames and quantitative participation: the participants were given a total of 12 days to work on the task. Within the framework of the introduction to the experiment, the participants declared themselves prepared to log onto the system at least once a day, to take note of the others ́contributions and to leave their own “trace” in the system. • Experimental Conditions: the experiment was carried out under two conditions. Groups with condition 1 were explicitly instructed to use the electronic negotiation tools in the consensus formation process (condition “with offer of e-negotiation” abbreviated to “with”), while groups with condition 2 (condition “without offer of e-negotiation” abbreviated to “without”) did not have the negotiation tool made available to them. The assignment of the groups to the conditions was random (in total 18 groups – 9 groups per condition). As two groups were not included in the final results (see GROUPS), there were nine groups with the condition “with” and seven with “without”. • Instructions: at the start of the experiment, the groups received their experiment instructions by e-mail. At the same time, the instructions were placed in the learning environment for everyone to see. As well as the afore-mentioned instructions for each experimental condition, the groups received instructions on organizational levels. To these belonged: the requirement to only carry out communication via the learning environment, to pass information on about how to start and to finish the participation in the experiment, to give hints on the organization of the process, to introduce themselves, to participate continuously as well as to inform the group members if a participation is not possible due to serious reasons. • Blind Interventions: during the 12 day experiment, the experiment leaders had no access to the virtual group work areas and were also unable to access the system’s logfiles. For this reason, blind interventions were sent to all participants via e-mail at pre-determined times. Table 3 gives an overview of these interventions. Data collection and analysis At the end of the experiment, the group work areas were closed to all participants. Immediately after the closing of the learning environment, all participants received an electronic questionnaire. This comprised, on one hand, questions on socio-demographic aspects, about familiarity with the learning environment Kolumbus2 and on experiences with collaboration within groups (virtual as well as face-to-face). On the other hand, they were asked to relate the elaborated group concept in their own words. These texts were then evaluated by six reviewers. The reviewers’ task was to compare the participants’ descriptions with respect to five aspects: • The similarity of both most similar descriptions (related to core statements) • The dissimilarity of both least similar descriptions • The degree of general similarity (related to all three descriptions of each group) • The degree of communality in the group concept • The novelty content / originality of the concepts. These types of questions were asked on the basis of the assumption, that similarities between the students’ description are an indicator of the degree of achieved convergence and knowledge integration. Each reviewer examined each one of the 16 groups. The results were given on a scale of one to ten (1= very low to 10= very high). After the quantitative analysis, which was based on the answers of the reviewers, the qualitative analysis of the group discourse with and without the offer of e-negotiation was carried out. Table 3. Overview of blind interventions Time Blind Intervention After 30 hours experimental time (ET) The participants were again advised of the specific instructions of each experimental condition. They were advised of the agreed participation frequency. After 6 days (ET) Participants were advised that half of the experimental time was up. They were also reminded of the task to produce a communal concept. After 11 days (ET) The last day of the project was announced. The instructions for the proper conclusion of the experiment were given. After 12 days (ET) The end of the experiment was announced and the learning environment was closed to all participants.